Adaptive Kernel Based Machine Learning Methods

During the support period July 1, 2011 June 30, 2012, seven research papers were published. They consist of three types: • Research that directly addresses the kernel selection problem in machine learning [1, 2]. • Research that closely relates to the fundamental issues of the proposed research of this grant [3, 4, 5, 6]. • Research that is in the general context of computational mathematics [7]. Paper [1] studies the construction of a refinement kernel for a given operator-valued reproducing kernel such that the vector-valued reproducing kernel Hilbert space of the refinement kernel contains that of the given kernel as a subspace. The study is motivated from the need of updating the current operator-valued reproducing kernel in multi-task learning when underfitting or overfitting occurs. Numerical simulations confirm that the established refinement kernel method is able to meet this need. Various characterizations are provided based on feature maps and vector-valued integral representations of operator-valued reproducing kernels. Concrete examples of refining translation invariant and finite Hilbert-Schmidt operator-valued reproducing kernels are provided. Other examples include refinement of Hessian of scalar-valued translation-invariant kernels and transformation kernels. Existence and properties of operator-valued reproducing kernels preserved during the refinement process are also investigated. Motivated by the importance of kernel-based methods for multi-task learning, we provide in [2] a complete characterization of multi-task finite rank kernels in terms of the positivity of what we call its associated characteristic operator. Consequently, we are led to establishing that every continuous multitask kernel, defined on a cube in an Euclidean space, not only can be uniformly approximated by multi-task polynomial kernels, but also can be extended as a multi-task kernel to all of the Euclidean space. Finally, we discuss the interpolation of multi-task kernels by multi-task finite rank kernels. Multiscale collocation methods are developed in [3] for solving a system of integral equations which is a reformulation of the Tikhonov-regularized second-kind equation of an ill-posed integral equation of the first kind. This problem is closely related to regularization problems in machine learning. Direct numerical solutions of the Tikhonov regularization equation require one to generate a matrix representation of the composition of the conjugate operator with its original integral operator. Generating such a matrix is computationally costly. To overcome this challenging computational issue, rather than directly solving the Tikhonov-regularized equation, we propose to solve an equivalent coupled system of integral equations. We apply a multiscale collocation method with a matrix compression strategy to discretize the system of integral equations and then use the multilevel augmentation method to solve the resulting discrete system. A priori and a posteriori